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•Statistical 



ABSTEAC^^^ ^ , , 

study Of college faculty has bed^ haapered by the 
irariation in ur^pes of institutions^ the difficulty of obtaining an. ^ 
ac<t:urate faculty saaple^ and poor response rat^ to large-scale ^ 
surveys. This i doi;iUflient^ examines various approaclhes\to conducting 
large*scale facttlty surveys^ and. describes a su Masted procedure to 
Bitigate sethodoliMical -problems* In developing the t^cossended 
proc^dure^ six, pilot tests were conducted to detersin« the 
feasibility of the nethodology^ the typids of letters t'^t should be 
sentf' the person to vhos these letters should be addressed^ and the 
anticipated response ^ate; ^ti^ pilot tests^4^i<^t<^d tha^t the college 
president is the best initial contact pointy tliat ^ liigh 
individual retsponse rat^ could be achieved througl^Jthe use' of an 
on*caspus facilitator* The procedures for sasple seie^SM^n and 
questionnaire administration are described, as is the applipation of 
the procedure to a aatiofivide sijKrvey of tiro-year college hasaMties 
instructors* The procedure desbrib^&^alloirs for cro^sAtabulatlons 
asong 1^espondents in various types ofslnstitutions vhiX^. saintaiiiing 
an accurate repipesentation of the universe /of institutions* 
(Author/NHM) ^ 
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^ Stddy of college faculty haa been limited by the variation In types 
ln8tltutlon$> the difficulty of .obtaining an accurate faculty a«mple> 
and the poor reaponae rate In large^acale aurveys^ Institutional variation 
demands that accurate Information can be. obtained otily If surveys are 
addressed to a broad ssmplis of colleges** The population of Institutions 
Includes private^ liberal atts-related colleges of fewer than 100 students* 
netf> public occupational and technical lnstltutes> multlcampiiis comprehen- 
sive colleges of than 30>000 students > and fteveral ^ther types in 
^ the various geographic^eglons* Before drawing Infere^es about faculty 
In colleges nationwide > th^^^Ssearcber must take care to assess Instructors 
In sll types of Institutions In >^oportlon to ^thelr numbers In , the pop- 
ulation as a whole* 

A iTQpresentatlve .sample of Colleges csn be drswn> but what of the 
fsculty within them? Sending surv^ forms to a college In wholeeale lots 
for distribution "to the fsciilty" Is rlsl^; the researcher never knows hgiu 
or If th^ were distributed. And asking someone on the csmpus to "ssmpXe'V 
a number of Instructors ill Irresponsible > especially If the researcher /needs/ 
a particular subgroup; the contact person may pick ^e first ten coming 
through t]h^ door* The researcher must undoubtedly adii^ss his quest/onnalres 
to specific Instructors^ but accurate faculty lists are not readllyj 
avsai^ble because the colleges do' not maintain faculty data uniformly* j 
Even tf^gh the catalog typically provides ni^es of full-time tea<^lng 
faculty>*lV la usually out-of-date* More lmportantly> the part-tftme and 



adjunct faculty are usually not Hated ^t all. Frequently employed at the 
last, minute > their namu may not be available until the term la under Way» 

A third problem-*the difficulty in obtaining responaea to surveys of i^ 
large population —has been well^doctimented. A conmon-'rand very undesirable*?-' 
practice la to mall out a huge number of queatlonnalre's and accept a small 
proportion of returns. Humeroua surveys reporting"" response rates ss tow 
aa 20 to 30 percent 4|*e found In the literature. One can only speculate 
on the systematic biases' among respondents in these samples*^ 

Som^ Investigators control for low rates of response by ^forcing" 
answers from ^ small number on non-resfkindents and comparing tliem with the 

it 

volitional reaponses. Others attempt to solve the problem by weighting the 
Respondents within categories > thus leveling the returns* These and others 
that might be mentioned are legitimate statistical stratagems. However If 
nearly all the population sampled can, be enticed to respond In the first 
place> the stratagem^ are rendered unnecessary. 

(} How to get responses? Most of the techniques described In the liter- 
ature are concerned with mailing procedurea IjHrhlch questionnaires are 
addressed to fche recipients. Astln and Panos (1969) found tliat auto-typed 
letters sent special delivery produced a higher response cheaper In terms 

of cost per respondent (former students) than did registered Biall> telephone 

% 

calls > or mimeographed letters- Rosamann and Astln (1974) compared fourteen 
^mailing techniques > each subtly different* and found that current and formeir 
college students were most efficiently surveyed using nonprofit outgoing 
postage* window envelopea; and bualneaa reply returns. The effects of offer* 
Ing cash rewaiaJa was noted by Dohrenwend (1970-1971) and Haclcler and Bourgette 
(1973); of i^^i^^ontact with reapondento by Walmsley (1973) and Parsons and 
^Medford (1972); of length of queatlonnalre by Champion and Sear (1969). 



The varioui procedures for increasing returns were summarized hy linsfcy 
(i975) whose survey of tHe literature found postcard reminders > pre-contact 
with respondent! by telephone* type of postage* cash rewards* and the type 
of organization Sponsoring the study all r^iportedly effective. 

Those who survey college faculty similarly attend to the problem of 
obtaining reliabla data. Some weight for noO*-respondents; the American 
Council on Educational national faculty survey (Bayer* 1973) obtained leaa 
than 50 percent returns and assigned categorical weightings to account for 
variation in response among such subgiroupa a^^octoral degree holders* 
professors in larffej research-oriented institutions* and so on. Others 
uS€ intensive follbwup procedures; by so doing* Xeslie received just under 
70 percent returns from a simple of Pennsylvania faculty (1974) and 
86 percent return from a sample of 100 Pennsylvania comminity college instruct- 
ors (1973). 

Better returns seem always to be obtained if an on-^site facilitator is 
there to deliver and retrieve the questionnaires "-a technique employed 
frequently by survey research organizations such as National Opinion Research 
Center and Field Research Corporation. Indeed* in a survey of two-^year 
college faculty members* Bushnell (1973) obtained a 90.9 percent response 
by having someone on each campus collect the forms. But his rate of return 
war severely inflated through his excluding from tabulation 2U of tha 92 
participating ^lieges because they returned fewer than 75 percent. In 
addition he allotted the on-^campus facilitator to select him own santple 
within prescribed lintlts. 

Charged with doing a nationwide survey of humanities instructors in 
two^-yaar colleges^ for the Hational Endowment for the Humanities* we were 
lad to develop and test e procedure for mitigating these problems. 
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The objfictivei of our investigation required a study group reprasentative 
of both full- and part*-time faculty meiQber& in the humanitie* and a compar- 
iion group of nonhumanitiea faculty. A further requirement wai that the 
group be large enough to permit croas-claaiification of information by 
jeveral variable! simultaneously. A mailed queitiotinaire was the only * 
method feasible within budget > but we felt it esaential that repreaentative* 
ness be asaured by following sound sampling principles and that reliability 
be maximized by obtaining a high rate of completed questionnaires. 

We decided on a two-stage sample—a broad Sample of colleges aelectied 
at, random within certain strata^ and a sample of the faculty within those 
colleges. The main stratification variables for the colleges would be type 
of control (pi^lic^or, i^ivate) and geographic locale because wa felt these 
wep»^^he mi^iti iits^it^ional differences "a^fectings^^^ faculty. Secondary 
v«ri^t»1llfti includ^^l q^llegVl enq^Kasis ;<comiirehei«ji!lve> technological^ liberal 
arts) / o^S^niz^tion Ooult and age. 

Iir orde"^ ^'lns0r«' co^^^ definition of tli^ populatfto tjte decided 
^o draif^ our own lia)^ of fjiculty tn^bers .teach^^ ^bu^ili^^^ in these colleges. 
The national EndcAhnent for the Humahit^s exclu^^il'' thi perf^ttfllng arts^^^from ' 



its purviBw. ^ Thus> we needed namee of i^^ioibXet" teiachlngb^oulfs^ Music 
Literature/Appreci^tion/History> butf not those wl]|^ taught ftelTforming music 
exclusively. Similarly we \Te^da<f teaci^e^s In Art Kistdry%nj)i.Af^£cle^ 
but not in trawing*^ Sculptura> ^Sr Deaiftn; :thea1b're HOttcify and Appreciation 
were in; Stagecraft, and Dr-ama were out* ""iiitera^ur^ in; Re&ding and 
Compoii£tian wete ouj^ We al#,D needed an cfn-<^ti^i[ faci%tdtar to send 
• necessai^ materials' to ua aivl, >tQ diat^rimit^ ^nd re^iVy^ the questionnaires 
so that; we would not ^i^^^t^^d with^tlje typicel ;loW respcmsa rate obtained 
in Individually-Tuallerf aurveysV . - ' : > ' ^ i 



ERIC 



*5- 



Several pilot teata were conducted to determine the feasibility of the 
methodology* the types of letters that should be addressed > the pattern 
of Interaction with the facilitators > and the responses we could anticipate* 
In one pilot test we sent the questionnaire to 29 faculty members selected 
at random from rosters In eight college catalogs* ISils procedur«> Including 
/one follow-up letter> yielded a predictably low return rate of 31 percent. 

Five additional pilot procedures were tried > each addressed to eight 
different colleges* Three of the pilots used different types of. letters 

r 

addressed to the president of the college > one was addressed to the dean of 
Instruction^ and In one we tnade a personal contact througltt phone or letter 
naming a tmit;ual acquaintance. That ls> In this latter procedure^ we Ident*- * 
Ifled a person whom we knew end who atso Icnew the president and who could 
be named as endorsing the project. * 

The pilot tests revealed that the president is the best IhltlalVontact 
point. The highest agreement to participate i»as obtained from the deans 
of Instruction^ but when Me followed through with the distribution of the 
questlonnal^s through the deans > the lowest rate of returns was revealed. 
In the pilot tests when we went through the presidents > only approximately 
half of them agreed to have their colleges participate^ but when they dld> 
from 88 to 94 percent of the faculty returned tha questionnaires* The lowest 
rate of return In this procedure was the one In which the personal contact 
was solicited through rdcoomendatlonsi Nevertheless the pilots did reveal 
that wa could anticipate a high 'Individual response rate through- the use of 
an on-cempus facilitator «nd that one-^half or more of the colleges Invited 
wouldj|fpartlclpate* 

^le next step was to determine the size of the sample* The Endowment 
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wantftd l>50(h returkis* Previous research had Indicated that approximately 
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20 percent of the full-time iMtructors In two-year colleges teach In 
the humanities. We had no Information on the part-timera» but we Buapecte4 ' 
a conilderably lover number. Anticipating 80 to 85 percent responae» 
therefore » we needed to tend out between 1»765 and 1»87$ iurveyi. Ife alio 
wanted a large enough sample of colleges— about 150^-to miximlxe the spread 
by type of college within feasible limits. \<rN 

The fli:at stsge In obtaining the sample of colleges cons^ted of draw- 
ing names from the 1975 Communlty t Junior > and Technical College Directory . 
Anticipating that about 60 perq&nt of the presidents would acquiesce lio 
our request to survey their faculty^ ye (decided to Invite 2A0 colleges 

Initially. The 1»184 colleges In the Directory are arrayed alphabetically 

J- 

by the 50 atatea. Randomisation by type of control 9nd geographic locale 
was Insured by starting at a random point and taking every fifth private 
and every fifth imbllc college. 

The second stage waa to develop the aample of humanltlaa Instructors t 
The colleges listed In the Directory show a total of 162^000 faculty. 
Aasumlng our sample of 150 collegea*-- about 12-1/2 percent of the total— to 
be proportionate by slze» we ahHclpated they would have 20^250 faculty 
(12^1/2 percent of the total). If 20 percent of the faculty were In the 
humanities^ our colleges would yield a pool of 4»050 names. However^ be*- 
csuse we expected that fewer of the part-tlmrs tauj^ht humanities we antici- 
pated that the colleges In the sample would ^ve batmen 3»500 and 3»750 
humanltlaa faculty members. Accordingly^ we decided that a large enough 
pool could be generated by aampllng one*-half of tha humanities instructors 
In each college. 

We ient letters Inviting participation^ asking for die names of a 
contact person to act as facilitator^ and asking that the facilitator 
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send a college catalog> a spring 1975 schedule of clasie8> and a fadulty 
roater If one TDore up-to-^date than the catalog Hating were available. 
We needed the catalog because the course descriptions would' toll ui which 
couraea properly fell within our purview. Thljs proved useful In auc^ areas^ 
as Anthropology where we wanted courses emphasizing Cultures of lMan> but not 

those focused qn physical Anthropology. Slmllarly> a course entitled ''Prln*' 

i 

clpjt^s of Geography" would be Irtclpded If It were described as a Cultural 
Geqgtaphy course > but not if It eittphaslzed scientific aspects* We heeded 
the coarse schedule so that we could draw the names only of the people who 
ware listed aa teaching thoae courses in spring 1975. And we needed the 
faculty roster In order to check for first names and cross-check Information 
such as deparbnental affiliation and chairperson status. 

A roster of humanities faculty for each college was generated by 
liatin$ all full-time and part-time instructors separately and picking ^ 
random one-half of each. In addition > we selected one-third as many 
depaiftment and division chairmen outside the humanities. Thus> if a college 
had a total of 20 full-time and four part-^tlme instructors > we would sample 
ten of the. fuXl-tiniers> two of .the part-^timers* "and four ^onhumanities 
Chairmen > yielding a total of 16 subjects for that college. This procedure 
demanded our reviewing every class schedule carefully> but we felt.it essential 
to produce, accurate rosters of people teaching one or more humanities courses 
in spring 1975. \ . 

We' had developed a .questionnaire including a lar^«N:ntmber of items 
arrayed in ten categories: demographic information; presenK^e preparation; 
preferences for curriculum and instruction; professional experiences; research 
orientation; concern for students; reference group identification; concern 
for the humanities; values; work satisfaction; and IFUnctional Potentiirl> a 

i 
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hypothetical construPt budLlt on pfy^odynamlc prlnclpleB of human JEtmctlon'* ^ 
Iti^. Wfi had pretested it In aeveral colleges In CtHfornle end had eskad 
numaroui profeeilanal jrsioclatlon headi and Individual Initructori In other 
parte of the country for iuggeitlona. The final veralon totaled 11 printed 
pages. ' - . . . 

After gulling the faculty lample for each college* we prepared packeti 
for dlitrlbutlon by the facilitator. Each pac^t Included a queitloimalre> 
en envelopje stamped *'Cbnfldentlal>*' lind e larger envelope addreised to the 
facilitator with the faculty member's name ot^^the outside. * Tl|e facilitator 
gave a packet to each named Inftructor. The reapondeht was Instructed to 
seal his Questionnaire Inside the confidential envelope> place It in the 
envelope addressed to the facllltator> and return It to^ him. The facilitator 
was' Instructed to check the respondent's name egelnst the roster we had 
provided > remove the outer envelope, and return only the sealed Inner- 
confidential envelope to us* In this way he could determine wlutf^ad not 
respondjbd> yet the Instructor's anonymity of response wss prot^cte^ beceuse 
the facilitator could not sea the completed questionnaires themselves. 
After the facilitator had retrieved the questionnaires > he returned them 
to us. If any were still outstanding* wa asked him to try to retrieve them. 
Contsct with the facllltatora was by both phone and letter. In no Instsnce 
did we contsct the respondents themselves* ^ 

One hundred flfty-slx colleges > nesrly exactly representative in terms 
of'control> local9> slze> age> emphasis/ snd organleatlon> participated In 
the atudy. The anticipated 20 percent of full-^tlme' faculty members teaching 
humanltlea proved to be accurate. Of the part-time faculty in the colleges 
Iniour study> 10**l/2 percent tsught in the humsnltles. The overall pool 
Included 2>384 questlonnslres ^ent;^ 2>008 were returiied> Including those 
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from the nonhumanities sapple* Questionnaires were retrieved froip 100 
^rcent of the faculty sampled in nearly two-thirds Qt the colleges* 
^rall> the response rate was 84 percent* Based on the checklists thet 
wer& returned from the facilitstors> we surmised that between four end 
five percent of the surveys were undelivereble because of inaccuracies in 
the schedules > last minute feculty substitutions | etc* Thus> we obtained 
a large pool of data with a minimal numtjer of ncmrespondents* 

Although the procedure demands extreme care and rigor in selecting the 
samples and j^rsuing the returns , we feel it is essentiel if generalizations 
to the universe of faculty members 4re to be made* A response rste.that 

•i 

finds only ero^n^ten percent of those receiving the questionnaires not 
returning them cen be assumed to be sn eccurate representation of the 
population without weighting, for respondent categories* And the stratification 

M 

of colleges ellws for cross-tebulations among respondents in vsrious types 
of institutions while maintaining eccurete representation of the universe 
of institutions* 
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